Maximum likelihood sequence estimation from the lattice viewpoint

It is well-known that the use of the Viterbi algorithm to implement a sequence estimator is an optimal way to remove the effect of intersymbol interference for digital transmission systems. However, such an implementation usually results in a very complicated receiver. In this thesis, we transform the problem of maximum likelihood sequence estimation into the problem of finding the closest lattice point. Some related lattice algorithms such as the basis reduction algorithms and the enumeration algorithms are analyzed and some improved versions are suggested. Then efficient algorithms finding the nearest lattice point are derived. Based on these lattice algorithms, simple but effective sequence estimators are proposed for the PAM systems and their complexities are analyzed. Under some mild assumptions, our algorithms have both polynomial space and time complexities, and are therefore much superior to the conventional Viterbi detectors. Simulation results on three different channels show that the performance of the new sequence estimators depend on the distance spectrum of the channel. But, general speaking, the performance approaches optimal as the size of the signal set and the signal-to-noise ratio increase. Finally, the extensions to other lattice-type modulation schemes and the impacts of the lattice viewpoint on the design of bandlimited transmission systems are discussed.